Fallbacks and retries

Fallbacks are on by default for every policy. A request that fails, times out or lands in a degraded provider region is retried on the next eligible model in the same call.

What counts as degraded

• A 5xx or 429 from the provider
• First-token latency above the policy's threshold for a two-minute window
• An error rate above 2% in the same window

Degradation is measured per provider region across all Prism traffic, so a region can be skipped before your own requests hit it.

Streaming

When a stream drops, the router resumes the completion on the next model with the tokens already delivered as context. The client sees one stream. The log records two legs, each with its own cost.

Limits

A request is retried at most three times. Each retry counts against the key's rate limit. Set fallbacks: off in the policy to disable them for a rule, or send x-prism-fallbacks: off per request.

Recovery

A recovered region is ramped back — 1%, 10%, then 100% of eligible traffic — over roughly ten minutes as long as its numbers hold.